Identifying the Unknown in User Space Memory

The research described in this thesis aims to improve the techniques used to analyse volatile memory, in particular user space memory, such that more generic techniques to identify unknown code such as malware can be produced. Current analysis techniques of user space memory are limited to a focus on the location of specific kernel objects. This focus on the identification of specific artifacts has also been applied in the detection of unknown code, which relies on the location of evidence created from the use of a particular malicious techniques. While such an approach allows the detection of unknown code, such as common malware, it allows a sophisticated attacker the ability to intentionally modify these artifacts to subvert detection. This research presents a generic method of detecting unknown code that does not rely on specific artifacts, allowing such malicious subversion to be overcome. Achieving this has required the creation of three individual contributions to improve the analysis of user space memory. The first is a model for identifying the common structural components that apply to the user space memory of all processes, such that the contents of user space memory can be better understood. The second is a model for distinguishing between code and data in memory, to facilitate the application of code or data specific analysis techniques. The final contribution is the technique for automatically and generically discriminating between known and unknown code in user space memory. This technique was capable of detecting the introduction of all malware samples examined.